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ABSTRACT 

strategies for assessing classroom teaching effectiveness at the college 
level. The assessments are generally ungraded activities which focus on 
current course work. This paper examines the following assessment activities 
within the context of teaching and technology: (1) "Muddiest Point -students 

jot down the muddiest point from a lecture, and the instructor addresses 
frequently mentioned points at the next lecture; (2) Empty Outlines- -students 
outline a portion of the lecture in a limited amount of time; (3) Minute 
Paper- -students record the most important thing learned during the class; (4) 
Double-Entry Journals- -students make notes about readings, then explain 
reactions to their notes; (5) Course-Related Self-Confidence 

Surveys- - students assess their self-confidence regarding specific skills; (6) 
"To Use or Not to Use" Analysis- -students prepare a recommendation for use of 
technology to solve a problem; (7) Procedure Brochures --students form a 
brochure providing directions for solution of a problem; (8) Keystroke 
Reports--one student works a problem using technology, while another records 
keystrokes used; (9) Technology Maps- -students provide a map/directory path 
for technological operations; (10) Technology Chain- -students cooperatively 
outline steps in a procedure that uses technology; (11) Pen Pals- -students 
write/e-mail a classmate about processes/concepts learned; and (12) Benefits 
Analysis- -students analyze benefits of the use of technology on that day. 
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Abstract: 

Angelo and Cross have developed strategies for assessing teaching effectiveness at the college level. As 
they have stated: "College instructors who have assumed that their students were learning what they were 
trying to teach them are regularly faced with disappointing evidence to the contrary when they grade tests 
and term papers." Angelo and Cross have developed some written "instant" feedback techniques which 
they call classroom assessment techniques. Generally using ungraded activities, the goal is to measure the 
"current state" of the classroom. We will examine a series of assessment techniques which are most 
useable in the technological classroom. 
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Introduction: 



Thomas A. Angelo and K. Patricia Cross have developed strategies for assessing classroom teaching 
effectiveness at the college level. As they have stated: "College instructors who have assumed that their 
students were learning what they were trying to teach them are regularly faced with disappointing evidence 
to the contrary when they grade tests and term papers." [1, p. 3] In order to address this issue, Angelo and 
Cross have developed some written "instant" feedback techniques which they call classroom assessment 
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techniques. The assessments are generally ungraded activities which focus on current course work. The 
goal is neither research in pedagogy nor evaluation of individual students. Rather, the goal is to measure 
the "current state" of the classroom. The goal of their work is to present assessment techniques which are 
brief, yet effective, and can be used quickly in the classroom. In order to understand the characteristics of 
these activities, let us first examine a "case study" of one of the most powerful of the classroom assessment 
techniques presented in Angelo’s and Cross’s book. After the case study, we will proceed to examine a 
variety of assessment activities within the context of teaching with technology. 

"The Muddiest Point" is an assessment technique which was originally developed by F. Mosteller at 
Harvard while teaching an undergraduate statistics course. [3] The basic strategy is to ask students to 
quickly jot down on a piece of paper the single "muddiest point" from the day’s lecture. The students 
submit the notes to the instructor, usually anonymously, at the end of the lecture. The instructor reports 
back to the students at the beginning of the next lecture by a) responding to the most mentioned one or two 
points, and b) briefly addressing as many others as possible and reminding students of additional sources of 
information. In a mathematics class, this is usually done by starting the next class with a well-chosen 
example problem, emphasizing the responses to the "muddiest point" notes while working it. Generally, 
there are no grades associated with this activity. The truly amazing aspect for this classroom assessment is 
that most often there are no more than two or three issues raised during this assessment. 

The muddiest point incorporates some of the most useful aspects of classroom assessment techniques. The 
greatest value of classroom assessment techniques comes from the combination of providing effective 
evaluation of on-going learning for the instructor and at the same time allowing the almost instant feedback 
from the assessment. Additionally, effective assessment techniques possess the following characteristics: 

• Improve instructor’s understanding of student needs and their perceptions of current material 

• Are immediately useable 

• Do not take up much class time 

• Are easy to administer 

• Are easy to analyze 

• Do not take inordinate time to analyze 

• Are flexible and can be useful for a variety of topics 

Typically, classroom assessments represent time saved in class since the instructor presents one or two 
examples and then can continue on with new material; sometimes, the questions can even be answered in 
the context of motivating the new material. From the students’ perspective, an immediate response to 
confusing questions allows a quick correction and students soon learn to take the process seriously. The 
whole process increases the trust between instructor and student; the instructor demonstrates concern for 
the student’s learning when implementing the technique and simultaneously receives information about the 
students’ on-going attention to the course. 

There are a wide variety of assessments presented in Angelo and Cross. [1, p. xiv] Each of them have a 
different "best use", a different learning purpose, and can be used by instructors in different manners. 
Longer techniques can be performed outside of class and may or may not include instructor grading. In 
each case, the major purposes for the techniques include assessment of: 

• Content comprehension 

• Analytic thinking 

• Student learning skills/processes 

• Students’ attitudes toward class activities 



The assessments chosen should be determined by the specific content/leaming to be measured and the 
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purposes of the instructor. In general, it is not a good idea to use too many different techniques in one 
semester. Student responses are more useful when the students are comfortable with a particular technique 
and understand it. 



Before addressing technological issues specifically, a few general comments about classroom assessment 
can be made. It is important to remember that the term "classroom assessment" refers to assessment of 
learning rather than grade assessment or educational research study. The goals are strictly to enhance "the 
continuous monitoring of students’ learning". [l,p. xiv] These techniques can be so simple that one often 
asks, "Can this really be useful?" or comments "But I already do this in a different setting." The results are 
still very revealing. "Completing the loop", that is getting the feedback to the student, is critical to 
maintaining the integrity of the process - students quit responding if there are no results. Finally, the 
assessment activities can be assigned in such a manner as to allow them to be graded; however, the most 
valuable are often the ungraded activities. 

When we speak of using technology in the classroom, we are speaking in the broadest sense: everything 
from calculators, computers and software to distance learning technologies. Although the argument can be 
made that most instruction involves the same pedagogical issues whether or not technology is involved in 
the instruction process, it is valuable to note that the dynamics of the teaching process can change with the 
introduction of technology. The two places where technological issues arise are in assessing whether the 
technology is providing an effective medium for the transfer of knowledge and whether the details of the 
use of the particular technology are being mastered. These two issues are at the heart of teaching with 
technology. Classroom assessment can provide effective analysis of each of these issues. To address the 
modification of existing classroom techniques, we will describe the general technique first and then 
suggest a modification specifically for technological issues. Techniques developed specifically for 
technology will then follow. 

ITqeI 

The Muddiest Point: 



As described earlier in this article, the Muddiest Point assessment is clearly useful in most learning 
situations; it can be particularly useful in combination with instruction using technologies. In a distance 
learning setting, the muddiest point activity combined with e-mail or with a coordinating teacher/facilitator 
at the remote site can restore some of the interaction between students and instructor which may have been 
compromised. If a course is using technology for instruction purposes, the instructor can phrase the 
"muddiest point" question to assess the students’ self-analysis of their ability to manipulate the technology. 
Finally, an exceptionally helpful question for use in the technological classroom is "What is the muddiest 
point about the connection between the process used on the calculator (computer, etc.) and the principle 
being studied today?" The Muddiest Point assessment is one of the most versatile of the assessments. 

ITqeI 



Empty Outlines: 



The Empty Outlines assessment [1, p. 138] has a name which provides a reasonably complete description 
of the technique. The instructor asks the student to outline some portion of the lecture in a limited amount 
of time. The student can be asked to start from scratch or the instructor may provide a partial outline of the 
lecture. This technique can measure student understanding and analysis or student listening. The first time 
the technique is used, the instructor may want to provide a partial outline before starting the lecture; this 
would allow the students to be prepared for the task at the end of the lecture. If results are mixed, the 
instructor may ask the students for feedback - did you not understand the concepts or could you not 
organize them? This technique has the added benefit that providing partial outlines can enhance students’ 
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understanding of the technique of studying through outlining. 

With respect to teaching with technology, there are additional applications of Empty Outlines. Outlines are 
particularly useful in understanding multi-step processes. Thus, this technique can be used to enhance the 
understanding of the steps in a technological process. Alternately, the instructor may ask the student to 
outline the concepts studied and then provide notations in the outline where technology helped 
demonstrate a particular fact or theory. Finally, when using technology as an instructional medium, the 
provision for the Empty Outline assessment before instruction can help the students at a remote site focus 
their attention on the purpose of the lecture and improve their concentration. 

Hop] 

Minute Paper: 

This technique, also known as the One-Minute Paper [5] or the Half-Sheet Response [4], is widely used in 
college classrooms. Once again the name is reasonably self-explanatory. Near the end of class, the 
instructor asks the students to record on a note-card (or a half-sheet of paper) "the most important thing 
you learned during this class." One may vary the questions to ask about items left unanswered, ethical 
consequences of the issues discussed, or once again one could address the question of student learning by 
asking them "What is the one thing from class today that you would most like to revisit?" The general 
benefit of this assessment is once again to establish a level of listening and "on-site" critical thinking on the 
part of the students. The first few times you try this, you will need more time and you will need to be 
patient. As students learn to expect the question, they will be more prepared and you will have altered their 
classroom learning style. 

The Minute Paper can be used to address specific technological issues by recasting the question. The 
students can be made aware of the value of the distance learning environment by asking "What aspect of 
this lecture befitted from the use of the technology available in our distance learning environment?" The 
instructor can evaluate the students’ understanding of the use of technology by asking "Which facts (or 
theory) have been demonstrated today through the use of technology? Could they have been as effectively 
demonstrated without the technology?" If one is particularly brave, one can recast the question as "Discuss 
how the technology improved your understanding or explain how it confused you." 



ITopI 



Double-Entry Journals: 

The general description of Double-Entry Journals is that students make notes about the beliefs or theories 
in their assigned course reading. As a second entry in the journal, the student explains his/her reactions to 
or questions about the notes in the first entry. This technique is modified in Angelo and Cross [1, p.263] 
from a technique discussed as dialectical notebooks by Ann Berthoff [2] Depending on the emphasis 
desired, one can direct these journal entries to focus on different specific aspects of the course work. These 
assessments can benefit from being graded as they require more of the student and instructor time and as 
presented must in general be done outside of class. 



For technological purposes, the Double-Entry Journal can be modified to emphasize the value of the 
particular technology in the leaming/teaching process. One can request the students divide the pages in half 
in their journal; on the left side they can record a sample problem and on the right side they can record the 
steps of the technological process (in words, not keystrokes) used in the solution of the problem. If the 
learning environment is a distance learning setting, the students can be requested to divide in half the paper 
used to record class notes. They can take their notes during class on the left side of the paper. When they 
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are away from the instructor, they can then review the notes and record questions or comments on the right 
side near the related class item. At the beginning of the next lecture session, the instructor can start the 
class with the students discussing a portion of the questions and comments. This can provide the students 
in a distance learning environment with a greater sense of communication with the class as a whole and 
with the instructor. 

[Top] 

Course-Related Self-Confidence Surveys: 

The goal of the Course-Related Self-Confidence Survey [1, p. 279] is to allow students who feel insecure 
in a specific skills context to establish their level of insecurity and to allow their instructor to monitor and 
hopefully moderate their insecurity. Requiring a little more preparation by the instructor, a survey is 
designed which requests that students respond to questions about their self-confidence regarding specific 
skills. Skills analyzed might involve public speaking skills, specific mathematical skills, or technical skills. 
The students rate their confidence on a scale of "none. . .low. . .medium. . .high". 

Clearly, this technique is very useful when a particular technology is used as a tool in a course. For 
example, if calculators or Mathematica are used in a calculus class, the instructor needs to know if the use 
of these technologies is actually interfering with the learning of the theory. If a lack of confidence towards 
specific processes can be identified, the instructor can direct instruction and practice to improve the 
students’ confidence. In a distance learning environment, this technique can be used during the initial few 
meetings to establish a better "comfort zone" for the students who hesitate to speak "on-camera". 

Instruction using technology, whether the use is incidental to the material, concurrent with the coursework, 
or the substance of the course, introduces its own variations in the learning process. In the discussion 
above, some "standard" learning assessment techniques were recast for use in a classroom with a 
technological component. The remaining techniques are ones that apply directly to the technological 
interfaces in the classroom - although they too could be modified to a more general setting. The three 
general attributes of teaching with technology which must be measured are: 

• The use of technology in support of other content 

• The use of technology as the content being assessed 

• The impact of distance learning/ general use of technology. 

In all cases, we continue to use the term technology to encompass all equipment of a technological nature — 
the broadest sense of the term. 



[Topi 



"To Use or Not to Use" Analysis: 



The instructor prepares a sample problem or the description of a situation. The student is requested to 
discuss reasons why technology is or is not needed to augment a theoretical analysis and to conclude with a 
definite recommendation for the use of technology in the solution to the problem. The solution to the 
problem is not included in the student work, but a numerical rating ("Rank on a scale of 1 to 10) 
determining the critical need for technology is to be made by the student. In other circumstances, the 
question might revolve around the choice between two differing technologies. Depending on the scope of 
the course, the decision can be based strictly on a student’s personal choices or it may include a more 
objective cost/benefit analysis in an advanced class. This technique is used primarily to analyze the use of 
technology in support of other content; depending on the detail, it may be done inside or outside of class. 
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IlQEl 

Procedure Brochures: 



The instructor prepares a sample problem or case study and asks the student to form a one-page brochure 
which would provide directions for the solution of that type of problem. The student is to include a brief 
outline of the steps to be taken, references to the appropriate pages in the text, and notations regarding 
where technology is to be incorporated into the process. This is an assignment to be completed outside 
class and is primarily used to assess the use of technology in support of other content. It can be recast for 
use when technology as the content is to be assessed by asking the students to create a brochure delineating 
the directions for "operating the technology" in the sample problem. 

Hop] 

Keystroke Reports: 

Often, an early barrier to teaching with technology is assessing whether or not the students have learned the 
right sequence of "keystroke" or operational steps to operate the technology. In this assessment, the 
students are divided into pairs and given a sample problem to work. One student works the problem using 
the technology; the other student records the exact steps/keystrokes used by the student. The assignment to 
pairs can be made permanent and the students can share the results of their work. For simple technologies, 
a few minutes of class time are more than sufficient for the assessment. For more complicated procedures, 
this may be an assignment for submission at a later date. This is an assessment which is directed towards 
assessing the students’ ability to perform procedures using a given technology. 

ITopI 

Technology Maps: 

As a student learns to use more advanced features of a particular technology - computer, calculator, etc. - 
the student may have difficulty finding a particular menu, sub-directory, or button. For this assessment, the 
instructor asks the student to provide a map or a directory path for several operations or actions. For 
example, in a class using calculators, the student might be asked to write out the path to the factorial 
button. In a class teaching a spreadsheet, the student might be asked to write out a map to performing a 
certain statistical function. These assessments of the student’s ability to perform certain technology tasks 
can be undertaken quickly in class. 

ITqbI 



Technology Chain: 



The goal for this assignment is to provide improved communication between students and to assess their 
ability to outline the steps in a procedure using technology. The instructor divides the class into pairs. 
Depending on the number of students in the class, the instructor hands a description of a process or 
procedure to one or more groups of students. Each group writes a description of the first step in the process 
of solving the problem and then hands the paper to another group. This group provides the second step in 
the process and then passes the paper to another group. The goal is to have each student group in the class 
provide at least one step. The instructor may take the finished products and compiles them into notes for 
the class, or the instructor may ask for the developed procedures to be read aloud. Do not do this 
assessment unless you plan to practice it often - students find the assessment difficult the first few times. 
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Structuring it as a series of relay races can sometimes help, depending on the character of the class. 

[Topi 
Pen Pals: 

Specifically for the distance learning setting, this assessment attempts to combine communication between 
students with a quick learning assessment. Each student is assigned a "pen pal" for the course. If there are 
students at two locations, the students should be paired so that one is from each location. The student is 
asked to write (or e-mail) one of the following: an outline of a particular process or concept, a paraphrase 
of a particular process, or a brief answer discussion of the most important concept of the day. If e-mail is 
not available, the teacher collects the papers during class and mails them to the facilitator of the other 
session for distribution to the pen pals. At the beginning of the next class, students can "read their mail" to 
the group as a whole. This can improve communication and comfort in a distance learning situation - 
although the logistics require careful execution. 

[Top] 

Benefits Analysis: 

On a day which concludes the use of technology for the analysis of a particular problem, or on random day 
in a distance learning environment, the instructor asks the student to comment on the pros and cons of the 
use of technology for that day and write a concluding benefits analysis of the use of technology on that day. 
If this activity is performed for a distance learning class, it is important to ask the student to include the 
personal benefits in having the class offered as distance learning. In this manner, the student can be 
reminded that although the use of technology may come at a price, in most cases it provides access to a 
learning environment which would otherwise be unavailable. Other than the One-Minute Paper, this may 
be the best method to assess student response to a distance learning setting; if the students never develop a 
perception that the final analysis is positive, then the structure of the course can be reconsidered. 

The goal of classroom assessment techniques is to "take the temperature" of the class learning environment 
more frequently than practical if using graded, extended activities. Since the use of technology introduces 
new variations into the classroom, it is even more important to understand the dynamics of the classroom. 
When you find an assessment technique that works for you, it becomes a very comfortable and certainly 
revealing practice which can help determine the direction of your teaching. 

[Top] 
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